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Executive Summary 


This report conveys the outcomes of a value-added analysis of Grade Level of Achievement 
(GLA) data at the provincial level. GLA data reported to Alberta Education is a teacher’s 
judgment of a students’ academic progress. GLA is based on the learner outcomes in a subject area 
after a course for a specific grade level has been completed and reflects the results from the full range 
of classroom assessments. Given the comprehensiveness of classroom-based assessment, analysis 
of GLA data can provide unique insights into factors that influence student achievement. GLA 
data provides teachers, principals, central office staff and Alberta Education with an additional 
tool to help inform and engage students and parents in the learning process, and to analyze and 
evaluate the achievement of different populations of students to ensure that their learning needs 
are better understood and met. When GLA is reported to parents and receiving teachers this 
information benefits students by supporting accurate and timely communication of achievement 
information as they move from one grade to the next or between schools. 


Alberta Education will use GLA data for program planning. The data is not used as part of the 
Accountability Pillar. The key purpose of reporting GLA is identifying students who are under- 
achieving, asking why and providing solutions both individually and systemically. 


Background 


GLA pilots to demonstrate and build GLA reporting capacity were run in 2003-04 and again 
with a larger set of schools in 2005-06. Detailed analysis of the outcomes of these pilots 
supported the reliability and validity of the initial GLA data collection initiatives (Alberta 
Education, 2005, 2007) and provided support for continuing with the implementation of the GLA 
initiative. 


This report describes the processes and outcomes associated with the 2006-07 Grade Level of 
Achievement (GLA) pilot data collection, data management and data analysis. Results of this 
and previous studies of GLA pilots will inform options for adjusting strategies for provincial 
implementation of the GLA reporting initiative. The current report demonstrates support for the 
four purposes for reporting GLA as defined in the GLA Handbook (Alberta Education, 2006:4), 
specifically: 

e to provide richer information at the system level (both jurisdictional and provincial) to 
inform effective practices to determine the impact of specific programs on student 
learning (e.g., English as a Second Language, special education) and to determine 
processes to further refine these programs; 

e asa catalyst within the school’s professional learning community to focus on individual 
student learning needs and interests; 

e to determine effective practices and strategies to foster higher levels of student 
achievement and confidence; and 

e to contribute to the data or evidence used to report student achievement to 
parents/guardians, fulfilling the school’s responsibility as outlined in the Guide to 
Education: ECS to Grade 12 in the section entitled Assessment as the Basis for 
Communicating Individual Student Achievement. 


Study Limitations 


Reporting of GLA to Alberta Education in 2006-07 was expected by one-third of the Grade 1-9 
schools in the province. This expectation was substantially exceeded with GLA data being 
submitted for the students in 60% of schools. The higher than expected number of students for 
whom GLA has been reported should still be treated with some caution, as there is still the 
potential for sampling bias which must be taken into consideration when reading this report. 


The data used in this report were extracted from the data warehouse at the end of September in 
order to support the timely production of this report. Some GLA data has since been updated in 
the warehouse, that is, reports viewed on the Extranet by jurisdiction personnel may have slight 
variances with the results shown in this report. 


Key Findings from the 2006-07 Analysis 
* The degree of jurisdiction participation in the 2006-07 pilot was much higher than expected. 
A participation rate of 33 % was expected; however, the data submitted represents 60% of 


schools. 


e The error rate for data submission was low with only 2.2% of the files submitted having 


errors. 


* Compared to the 2005-06 data, the 2006-07 data demonstrates similar results for the total 
cohort and for specific sub-groupings of GLA data. Patterns of data distribution observed in 
2005-06 with 82,000 students were also observed in the 2006-07 data. 

e The data analysis demonstrates many interesting relationships. For example: 


oO 


The large difference between Math 9 Provincial Achievement Test data and GLA 
data continues to be evident. 

A much higher percentage of students coded as gifted are assessed above grade 
level in Math than in English Language Arts or French Language Arts. 

When comparing foreign-born and Canadian-born students, both groups of 
students perform similarly in Mathematics but in English Language Arts, 
Canadian-born students are at an advantage. 

Although a serendipitous finding, there is a disproportionate ratio of male: female 
severely coded students. There are nearly four times as many males as females 
coded as severely disabled. 

When comparing GLA and PAT data, females outperform males in GLA to a 
statistically significant degree in both subjects in nearly all enrolled grades. The 
reverse is seen on the Math PAT results with males outperforming females to a 
statistically significant degree. In English Language Arts, females are 
outperforming males on the PATs. 

A greater proportion of high mobility students (i.e. those who have changed 
schools more frequently than other students) have a GLA below their enrolled 
grade level. 


e When examining mcan GLA compared to enrolled grade, interesting patterns emerge: 


° 


Foreign-born and Canadian-born ESL students’ GLA distributions are much 
closer together this year than was seen in the 2005-06 data. 


° 


Foreign-born ESL students seem to have a GLA on par with their enrolled grade 
in Mathematics up until Grade 5. In English Language Arts the two measures 
diverge as early as Grade 2. 


¢ The age effect! is apparent in English Language Arts in Grades | through 5. After Grade 5, 
the age effect tapers off and is no longer apparent. This relationship is more pronounced in 
the 2006-07 GLA data than in 2005-06, possibly due to the larger number of students in the 
2006-07 data. 

e There are moderate strength correlations between PATs and GLA (Grades 3, 6 and 9). This 
demonstrates a reasonable degree of concurrent validity of the GLA data. 

e Using GLA and PAT information, GLA can provide important information for students that 
would not otherwise be available for students in grades not tested by PATs. 


How will GLA Information be used? 


e The GLA data is intended for use by Alberta Education, school jurisdictions and schools 
to better inform parents of their child’s progress. 


e Examples of Alberta Education data usage are: 


2 


° 


To assist in planning improvements to provincial initiatives or programs and for 
curriculum evaluation. 

To conduct further research into understanding how factors such as student 
gender, age, mobility and socio-economic variables influence student 
achievement and how interventions strategies can mitigate such negative effects. 
This research will be shared with jurisdiction staff to inform local strategies for 
improving student achievement. 


e School jurisdiction and school usage include: 


oO 


GLA data will help to ensure students and parents know and appreciate the grade 
level of the program of studies that the student has achieved. This information 
can also assist grade or school transitions so that programs are planned based on 
the demonstrated achievement to date. 


Using GLA data as a benchmark, jurisdictions may wish to compare the data in 
school and jurisdiction GLA reports over time or in relationship to provincial 
norms. This can support conversations in professional learning communities 
regarding promising practices that have been demonstrated to improve student 
achievement in specific settings. 


e In addition, GLA data collected by Alberta Education will be available upon request to 
schools as a backup if a student’s record is delayed in transit or lost. This service will 


' Age effect is defined as older students in a grade tending to have higher average test scores than the younger students in that 
same grade when measured by the z-score of average PAT results for each birth month group (Alberta Learning, 2001). 


ill 


help teachers ensure their instruction is geared to the student’s instructional level and will 
help the student’s transition to their new school. 


Next Steps in Implementation 


Current plans call for submission of GLA data in Language Arts and Mathematics by all Grade 
1-9 schools in the province at the end of the 2007-08 school year. Plans are on track for 
receiving this data with zero errors. Production of GLA reports will provide schools and 
jurisdictions with analysis of their GLA data in relationship to their students’ age, gender, 
mobility and a range of socio-economic variable that will assist staff in gaining additional insight 
into how these variables influence their students’ achievement. These reports will be available 
via the Extranet to the submitting schools and jurisdictions by mid-October 2008. The provincial 
report summarizing the 2007-08 GLA data will have a strong focus on how well various 
programs are serving Alberta students. 


Introduction 


This report describes the processes and outcomes for the 2006-07 Grade Level of Achievement 
(GLA) pilot data collection, data management and data analysis. The report is also intended to 
define the provincial context supporting the four purposes for reporting GLA as defined in the 
GLA Handbook (Alberta Education, 2006:4), specifically: 

e to provide richer information at the system level (both jurisdictional and provincial) to 
inform effective practices to determine the impact of specific programs on student 
learning (c.g., English as a Second Language, special education) and to determine 
processes to further refine these programs; 

e asa catalyst within the school’s professional learning community to focus on individual 
student learning needs and interests; 

e to determine effective practices and strategies to foster higher levels of student 
achievement and confidence; and 

e to contribute to the data or evidence used to report student achievement to 
parents/guardians, fulfilling the school’s responsibility as outlined in the Guide to 
Education: ECS to Grade 12 in the section entitled Assessment as the Basis for 
Communicating Individual Student Achievement. 


Limitations of the Data 
When analyzing the data, the following limitations were noted: 


¢ In 2006-07, a much bigger sample than expected was submitted that is somewhat 
representative of the full population of Grade 1-9 students, however, some degree of 
sampling bias should be assumed. 


e Generalizations should be considered in relationship to the estimated confidence intervals 
detailed on page 7. 


GLA 2007 Data Collection 


Given that this was still a pilot year, a remarkably low number of data transmission errors 
occurred during this year’s GLA data collection. Of a total of 228,150 GLA records submitted, 
there were 5,023 errors received (2.2%). The following table lists the number of errors by 
jurisdiction and the number of schools affected by these errors. 


The following list shows the types of errors that occurred as well as the number and percent of 
each type of error. 


Number | Percent | 


Grade English Language Arts Introduced column incorrectly filled in 2,377 47.3% 
for FLA students 


Total 


The Edulink software and manual used to transmit GLA data to Alberta Education has been 
upgraded to identify solutions to errors that exist in the data files when they are transmitted to 
Alberta Education. The GLA-Edulink Manual will be revised to require errors be corrected 
before the deadline for GLA data submissions. This step is expected to reduce the error rate to 
0%. 


The individual student records having errors were not included in the 2006-07 data throughout 
this report nor in the school and jurisdiction reports available on the Extranet. School authorities 
were advised of the specific error reports so they could accommodate the error rate in their 
interpretation of the value added GLA reports provided back to the jurisdiction and schools that 
submitted GLA data. A total of 5,023 files were excluded from the data analysis due to errors. 


"In this particular jurisdiction the bulk of the errors occurred in three schools, the remainder of schools had only one 
or two errors per school. 


Adjustments for GLA 2008 Data Collection 


Quality control measures have been built into the Edulink reporting system. In 2007-08, 
jurisdictions submitting files that contain errors will be sent an automated email indicating the 
files containing errors. Files containing errors are to be amended and re-sent via Edulink by July 
7, 2008. 


Description of GLA Data 


A total of 923 schools from 71 authorities submitted useable Grade Level of Achievement data, 
reporting for 220,682 students, 3,380 of whom were not on a graded curriculum. The fields 
collected are as follows: 


All Students: 

e student name (surname and given name), 

e Alberta Student Number, and 

e enrolled grade, (defined as the grade to which the student was assigned) . 


GLA was collected for students on a graded curriculum as defined in the Alberta programs of 
study, in the following fields where applicable: 


e GLA in English Language Arts 

e GLA in French Language Arts - (French as the Language of instruction or Immersion 
students) 

e GLA in Mathematics 

e Grade English Language Arts Introduced 


Grade Level of Achievement in 2006-07 is defined as the grade level expressed as a whole 
number in relationship to the learning outcomes defined in the program of studies that teachers 
judged the student to have achieved at the end of the school year. A GLA Handbook (Alberta 
Education, 2006) was developed and distributed in the 2005-06 school year to facilitate pilot 
school participation in GLA reporting. 


The GLA Handbook encourages teachers to consider GLA assessment in relationship to the full 
range of formative and summative assessment information available to them over the course of 
the school year in making a professional judgment of the student’s grade level of achievement. 


Students not on a graded curriculum also had data submitted. “Not on a Graded Curriculum” 
was meant to indicate that the student’s program was restricted to learning outcomes that were 
significantly different from the provincial curriculum defined in the program of studies and were 
specifically selected to meet the student’s special needs as defined in the Standards for Special 
Education (Alberta Learning, 2004). The information collected was teachers’ ratings of 
students’ learning outcomes in three areas: communication skills, functional skills and academic 
readiness skills. “Communication skills” refer to the development of expressive and/or receptive 
communication. This could be verbal communication and/or alternative modes of 
communication. “Functional skills” refer to skills that would assist the student in developing 


independence in the home, school and community. “Academic readiness skills” refer to skills 
that would prepare the student for learning outcomes in the programs of study. 


Alberta Education staff used the Alberta Student Number to append data fields such as 
Provincial Achievement Test (PAT) results (both raw scores and achievement levels), student 
age, gender, number of school registrations, any additional special needs codes associated with 
the student, and starting date. Individual student identifiers were replaced with a discrete GLA 
data ID, leaving no personal identifiers in the dataset used in producing this report. 


All Charter School Authorities were required to report GLA in Language Arts and Math for all 
students in Grades | to 9. Jurisdictions were required to report GLA in Language Arts and Math 
for a minimum of one third of schools in Grades | to 9. Many jurisdictions reported more than 
one third of schools in the 2006-07 school year. Seventy-one authorities submitted valid GLA 
data in the 2006-07 school year. The table below outlines the number of schools submitting 
GLA data in each jurisdiction. 


Table | - Jurisdictions Reporting GLA 


Percent of 
Schools 


Number of 
Schools 


Number of 
Schools 


Jurisdictions Reporting 


Reporting 
| AlmadinaSchool Society —“‘“‘;é‘*dYSC“‘(‘®’CSC§CUYdCL:SC(SNSNCN§NNNN VJ} ——=«S100.0% | 
|AuroraSchoolLtd. ——“‘“SSCCOC*LCOC*#C( jOWdYS:COOCOC#(_C(#D*L «SC«#dO.0% | 
| Battle River Regional DivisionNo.31 sid Cid. 
| Calgary Arts Academy Society Cid Ci S«i002.0% 
| Calgary Girls'School Society —“*‘dRSC“‘CSSOCNLOCOCTCYL C1000 | 
| Calgary Roman Catholic Separate School District No.1 | 90] 32 | 35.6% 
| Calgary School DistrictNo.19 —‘idT:SCSS*C*=i‘i GT C*d9G | «100.0% | 
| Calgary Science School Society CC‘ CTC «100.0% | 
| Canadian Rockies Regional DivisionNo.12_ | CG] CO] «i100.0% | 
CAPE - Centre for Academic and Personal Excellence ies: aS 100.0% 
| Chinook's Edge School DivisionNo.73_—idTC TCT 29.7% | 
No. 3 
| Clearview School DivisionNo.71—CidTEC (CT 55«O% 
East Central Alberta Catholic Separate Schools Regional is. Sa. 
Division No. 16 
| East Central Francophone Education RegionNo.3 | 4 | 25.0% 
| Edmonton Catholic Separate School DistrictNo.7_ | 80 | 38.8% | 
| Elk Island Catholic Separate Regional DivisionNo.41_ [| —sdIG S| 8.3% | 
| Elk Island Public Schools Regional DivisionNo.14_ | 40] 9 | «987% 
| Evergreen Catholic Separate Regional DivisionNo.2 | 8 | | 25.0% | 


Percent of 
Schools 


Number of | Number of 
Schools Schools 


Fort McMurray Public School District No. 2833 12 


Fort McMurray Roman Catholic Separate School District 
No. 32 


Fort Vermilion School Division No. 52 


Foundations for the Future Charter Academy Charter 
School Socie 


Golden Hills School Division No. 75 4 


Jurisdictions Reporting 


p rting 
91.7% 
33.3% 


27.8% 
100.0% 


50.0% 
No. 28 

50.0% 

95.5% 
5.3% 
0.0% 


Grande Prairie School District No. 2357 
Grasslands Regional Division No. 6 


Greater North Central Francophone Education Region No. 1 
2 

Greater Southern Public Francophone Education Region 

No. 4 

Greater Southern Separate Catholic Francophone 

Education Region No. 4 
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High Prairie School Division No. 48 


Horizon School Division No. 67 
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18.2% 
Lakeland Roman Catholic Separate School District No 100.0% 
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100.0% 
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30.0% 
37.5% 
100.0% 
100.0% 
35.0% 
96.0% 
27.3% 
33.3% 
62.5% 


Medicine Hat Catholic Separate Regional Division No. 20 
Medicine Hat School District No. 76 


Holy Spirit Roman Catholic Separate Regional Division 
No. 4 


Northland School Division No. 61 
Northwest Francophone Education Region No. 1 
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Greater St. Albert Catholic Regional Division No. 29 


Lethbridge School District No. 51 
Living Waters Catholic Regional Division No. 42 


Grande Yellowhead Regional Division No. 35 
Livingstone Range School Division No. 68 


Holy Family Catholic Regional Division No. 37 


Mother Earth's Children's Charter School Socie 
New Horizons Charter School Socie 
Northern Gateway Regional Division No. 10 


i l 
Northern Lights School Division No. 69 
l 


Peace Wapiti School Division No. 76 


| Peace River School DivisionNo.10 | Cd | 33.3% 
i 24.1% 

25.0% 
| Prairie Land Regional Division No.25_ | Sd 16.7% 
97.1% 
26.7% 
27.6% 
Rocky View School Division No. 41 16 43.2% 


Jurisdictions Reporting Number of | Number of | Percent of 


Schools Schools Schools 

(K-12 Reporting Reporting 
| St. Paul Education Regional DivisionNo.1 | GT 56.3% | 

St. Thomas Aquinas Roman Catholic Separate Regional 7 

| Suzuki Charter School Society | (100.0% | 
| Westmount Charter School Society | 100.0% | 
| Westwind School DivisionNo.74 | 13.8% | 
| Wild Rose School DivisionNo.66 | GST 813% 
61.9%" 


* Provincially, 60% of all schools submitted GLA data. This table has excluded Parkland School Division that was 
exempted for 2006-07, Moberly Hall Charter school which closed, and Boyle Street Charter school that had no 
grade 1-9 enrolment from the school counts resulting in a higher percentage of schools submitting GLA data. 


Grade Level of Achievement — Summary of Results 


Students on a Graded Curriculum 


There were 217,302 students on a graded curriculum included in this sample for the 2006-07 
school year. The students are roughly evenly distributed by enrolled grade with approximately 
11% of the students in each grade cohort. The table below shows the distribution of the GLA 
sample data by enrolled grade (Grades | to 9) compared to the Provincial’ distribution. 


Table 2 - Enrolled Grade Distribution 


Enrolled Grade Distribution 


Percent of Province Percent of 95% 
Enrolled Frequency Total Total Confidence 
Grade Interval 
l 23,692 10.9% 42,176 10.6%  +/-0.47% 
2 23,929 11.0% 42,493 10.7%  +/-0.47% 
3 23,896 11.0% 42,656 10.7%  +/-0.47% 
4 23,954 11.0% 42,597 10.7%  +/-0.47% 
5 23,839 11.0% 43,858 11.0% +/-0.47% 
6 24,582 11.3% 45,050 11.3% +/-0.46% 
‘/ 24,240 11.2% 45,544 11.5% +/-0.47% 
8 24,267 11.2% 45,790 11.5% +/-0.46% 
9 24,903 11.5% 47,014 11.8%  +/- 0.45% 
Total 217,302 100.0% 397,178 100.0% —+/- 0.15% 


Students could have been coded as severely disabled, mild/moderately disabled, gifted or ESL. 
Those students who were not coded as any of those categories are termed ‘non-coded’. The non- 
coded students make up the largest proportion of the sample as shown in the chart below. 


* In Tables 2 and 3, the “Province” columns refer to the data for the entire province. In the remainder of the report, 
proviticial comparisons refer to the provincial sample of GLA data. 

The estimated confidence interval is expressed as a percentage and indicates how confident the researcher wants to 
be (e.g. 95% confident) that the data collected accurately represents the results of the entire population. The 
objective is to obtain a sample large enough to ensure a 95% confidence that the results will be within +/- 3% 
around the mean. In this report sample sizes are very large and provide a high degree of confidence that sampling 
error is minimal. 


Table 3- Types of Student Codes 


Type of Student Codes 


GLA Percent of Provincial Percent of 95% 
Totals GLA Total Totals Provincial Confidence 
Total Interval 

Non-Coded 170,637 77.9% 313,612 79.0% +/- 0.17 
Severe Disabilities 5,303 2.4% 12,576 3.2% +/- 0.86 
Mild/Moderate Disabilities 16,769 7.7% 30,835 7.8% +/- 0.56 
Gifted 3,314 1.5% 5,221 1.3% +/- 1.31 
ESL —Canadian-born 11,795 5.4% 18,690 4.7% +/- 0.69 
ESL — Foreign-born 11,139 5.1% 16,244 4.1% +/- 0.71 
Total 218,957* 100.0% 397,178 100.0% +/- 0.13 


*This is higher than the total in Table 2 because some students have double codes. 


The distribution of students in each of the GLA results categories by subject is shown in the table 
below. 


Table 4 - All Students, Provincial 


All Students — Province 
Mathematics English Language Arts French Language Arts 


Gt 
students j — enrolled (%) students | — enrolled (%) ~ students enrolled (%) 
? 
Prom ram | tomo |r 


Table 5 - Students Non Coded, Provincial 


Students non coded (as mild/moderate, severe, gifted, or ESL) — Province 


an ie enrolled (%) students enrolled (%) students enrolled (%) — 


es 


° GLA NA refers to missing data, a “not applicable” situation, or “not available.” 


Severe Disabilities 

There were 5,303 students coded as severely disabled who had GLA reported for English 
Language Arts and Mathematics. There were 69 students coded as severely disabled who had 
GLA reported for French Language Arts. The chart below presents students having a severe 
disability code and their grade level of achievement. 


Table 6 - Severe Disabilities, Provincial 


Students coded as having Severe Disabilities — Provincial 


English Language Arts French Language Arts 


Number of Percent of total | Numberof | | Percent of tot Nts, 
students enrolled (%) students ven ‘ enrolled (%) 
5% 


a) 


More than half of students with a severe disability enrolled in Mathematics have a GLA equal to 
their enrolled grade (55.1%). In Language Arts 53.5% of students have a GLA equal to their 
enrolled grade. Mathematics and Language Arts are fairly similar in their GLA distribution. A 


very low proportion of French Immersion or French as the language of instruction students were 
coded as severely disabled. 


The chart below provides information on all students who were on a graded curriculum and were 
coded as severely disabled, by their disability type. The majority of students coded with a severe 
disability are those with an emotional/behavioural or physical/medical disability. Among the 
different types of severe disabilities, the distribution of students among GLA categories varies 
more widely than the overall severe disability distribution (see above table). Knowing the type 
of severe disability may aid in understanding the student distribution throughout GLA categories. 


English Language Arts 
. %ator | Ybelow | %GLA~ 
aboye grade | grade level 


__level 
3% ; 1; a 


56 
‘Blininss Sid | tae | m9 [ 200) 15m] mam] 250m | 29%, 


Note: 1 ECS student was not shown on this table. The student is included in the remainder of the report as they are enrolled in a grade. 


Table 8 - Type of Severe Disability by Gender 


Type of Severe Disability by Gender 
_ Gender 


% GLA 
NA 


i 


grade 


16.7% 33.3% 
Pf Rem] | 9 | soe | non] oan] seen] — axon | sn] 
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Note: ] ECS student was not shown on this table, The student is included in the remainder of the report as they are enrolled in a grade. 
* Asterisk indicates a statistically significant difference between genders for the GLA categories indicated, Differences are significant at the 0.05 level, 
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When the students coded as having a severe disability are shown by gender, the large difference 
between genders becomes apparent. A large proportion (77.4%) of these students were males, 
most of whom were students with an emotional/behavioural disability. 


Mild/Moderate Disabilities 

There were 16,769 students in the 2006-07 sample having mild or moderate disability codes in 
English Language Arts and Mathematics. In French Language Arts, 369 students were coded as 
having a mild or moderate disability. The chart below shows the mild or moderate students’ 
distribution across GLA categories. 


Table 9 - Mild/Moderate Disabilities 


Students coded as having Mild/Moderate Disabilitics — Provincial 


| Mathematics | English Language Arts French Language Arts 


| Percent of total . Percent of total r of Percent of total 
enrolled (%) ¥ nts - enrolled(%) =f. ; | enrolled (%) 


me 


GLA below enrolled grade 


GLA equal to enrolled grade 


GLA above enrolled grade 


GLA NA 


In both Mathematics and Language Arts, a slim majority of students have a GLA equal to their 
enrolled grade. 
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The chart below shows the breakdown of students on a graded curriculum having a mild or 
moderate disability by the type of disability. The largest proportion of students coded with a 
mild/moderate disability has a mild cognitive disability or a learning disability. Students having 
a mild or moderate cognitive disability compose the largest proportion of students attaining 
below their grade level. As in the severe disability chart, students who were ‘at or above grade 
level’ were combined into a single category due to the small number of students who were 


achieving ‘above grade level’. 


Table 10 - Type of Mild/Moderate Disability 


Type of Mild/Moderate Disabilit English Language Arts 


| Frequency Percentage % at or % below | % GLANA % at or % GLA 
of Total above grade | grade level "9 above grade grade level NA 
2 level level 


Mild Cognitive Disabilit 68.7% 9.7% 69.7% | ___9.2% 
64.5% 23.7% 67.7% 22.6% 


E 


FauhiieDistiiy | 82] sim] ssa] aaa] 22% | nom | amare] 2.0% | 


Note: 2 students coded as ECS Developmentally Immature were removed from this analysis, They have been included in the rest of the report. 


The following chart shows each mild/moderate disability type by gender. 


1] 


Table 11 - Type of Mild/Moderate Disability by Gender 


Type of Mild/Moderate Disability by Gender English Language Arts 
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Note: 2 students coded as ECS Developmentally Immature were removed from this analysis. They have been included in the rest of the report. 
* Asterisk indicates a statistically significant difference between genders. Diffcrences are significant at the 0.05 level. 


Table 11 illustrates the gender differences in GLA categories across the mild/moderate disability 
types. The gender difference is most noticeable for students having a physical or medical 
disability. 


Gifted 

In this sample of students, 3,314 were coded as being gifted in English Language Arts or 
Mathematics. There were 156 students in French Language Arts coded as gifted. The chart 
below shows the grade level of achievement distributions for students coded as gifted. 
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Table 12 - Gifted Students, Provincial 
Students coded as gifted — Provincial 


| | Mathematics __|_ English Language Arts__| French Language Arts 


enrolled (%) =| a se nrolle 
[Tot 


The general assumption with gifted students is that they tend to achieve and perform better than 
the population of students as a whole. According to the above frequency table, we can sec that 
the majority of gifted students are performing equal to their enrolled grade level, and slightly 
more that 2% of students are ‘below grade level’ for each subject. Gifted students are 
performing better in Mathematics than in English Language Arts with approximately 7% more 
students having a GLA ‘above grade level’ in Math. 


(%) 


English as a Second Language 
For English as a Second Language (ESL) students, there were two groups of student codes. The 


first are ESL students coded as Canadian-born (303). Of this group of students there were 
11,795 in English Language Arts and Mathematics and 353 students in French Language Arts. 
For ESL students who are coded as foreign-born (301), there were 11,139 in English Language 
Arts and Mathematics and 242 in French Language Arts. The distribution of both groups’ GLAs 
is shown in the charts below. 


Table 13 - Canadian-born ESL Students, Provincial 


Students coded as English as a Second Language — Canadians-born (303) - Provincial 
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Table 14 - Foreign-born ESL Students, Provincial 


Students coded as English as a Second Lan ¢ - Foreign-born (301) - Provincial 


| Mathematics | 


Percent of total | Percent of total 
enrolled (%) nts enrolled (%) 


The above tables show that both Canadian-born and Foreign-born ESL students follow a similar 
pattern to one another. Approximately 83-84% of each ESL group is attaining a GLA that is 
equal to or above enrolled grade in Math and about 72-78% are equal to or above grade level in 
ELA. 


Gender 

Students’ GLA was broken down by gender in order to observe any patterns that may emerge. In 
this sample of students there was a larger number of males than females. The charts below show 
students’ GLA by gender in each of the three subjects. 


Table 15 - Gender, Mathematics, Provincial 


Gender - Provincial Mathematics 


Percent of total 
~~ enrolled (%) 


Note: All of the above observed relationships were significant when measured by Chi square. 


Table 16 - Gender, English Language Arts, Provincial 


Gender - Provincial English Language Arts 


“Percent of total 
enrolled (%) 


100.0% 111,372 100.0% 


Note: All of the above observed relationships were significant when measured by Chi square. 
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Table 17 - Gender, French Language Arts, Provincial 


Gender - Provincial 


of _ Percent of total 
students | _ enrolled (%) 


Note: All of the above observed relationships were significant when measured by Chi square. 


The three tables above show the frequency of female and male students in each of the GLA 
categories in Mathematics, English Language Arts, and French Language Arts. When testing for 
independence between genders, Chi-squared’ was used. There is a statistically significant 
difference between the genders in both Math and English Language Arts. 


Pope, Wentzel and Cammaert (2003) conducted a study which showed that larger gender 
differences were found in school-awarded marks than in the diploma exam marks in favour of 
females in almost all diploma courses. Math courses (both Math 30 Pure and Applied) were 
among those subjects where boys, while being outperformed by girls in school-awarded marks, 
did better than girls in diploma exam marks. Therefore, the question of gender differences 
between GLA and PAT results were of interest in the current report. 


The following tables illustrate the differences in GLA and PATs across the enrolled grades for 
each gender. It is quite interesting to note that, similar to the 2003 study, on GLAs, females 
outperform males to a statistically significant degree in both subjects in nearly all enrolled 
grades. The reverse is seen on the Math PAT results with males outperforming females to a 
Statistically significant degree. However, in English Language Arts, females are outperforming 
males on the PATs. These findings point to a potential area for future research to investigate 
gender differentials and the causal factors associated with these differences. 


” Chi square is defined as “A nonparametric procedure for testing whether the observed frequencies of scores in 
different categories of a variable differ from the theoretically predicted frequencies” (Harris, 1995). 


15 


Table 18 - English Language Arts T-Tests 
English Language Arts GLA T-Tests 


Enrolled Gender 05-06 
Grade N 
] F 3,562 
M 3,518 
2 F 4,161 
M 4,356 
3 F 4,236 
M 4,511 
4 F 4,281 
M . 4,633 
5 F 4,535 
M 4,664 
6 T ES 4,280 
M 4,689 
7 F 4,127 
M 4,350 
8 F 4,444 
M 4,489 
9 F 4,466 
M 4,571 


05-06 
Mean 
GLA 
1.01 
1.01 
1.93* 
1.89* 
2.89* 
2.84* 


06-07 
N 


11,525 
12,167 
11,704 


2225 


11,641 
12,255 


3.86* 11,608 


3:79" 
4.81* 
4.75* 
5.80* 
PY AF 
6.87* 
6.75* 
7.88* 
TATE 
8.87* 
8.77* 


12,346 
11,440 
12,399 


12,114 


12,468 
11,790 
12,450 
11,849" 
12,418 
12,259 
12.644 


06-07 


English Language Arts PAT T-Tests 


Enrolled Gender 05-06 05-06 06-07 06-07 Mean 
Grade N Mean PAT N PAT Score 
Score 
3. EF 4,088 71.03* — 10,790 69.72" 
M 4,244 68.50* 11,537 66.72* 
& fr _ 4 
6 F 4,039 68.36" 11.451 69.79%" 
. M 4,304 64.90" 11,923 © 65.40* 
és — all “ 
9 FE 4,154 71.42* 11,337 68.36* 
M 4,139 66.77* 11,742 63.25* 


* indicates a statistically significant difference between males and females. 


Table 19 - Math T-Tests 


Mathematics GLA T-Tests 


Enrolled Gender 05-06 05-06 06-07 06-07 
Grade N Mean WN Mean 
GLA GLA 
1 F 4,024 1.01 11,525 0.95* 
M 4,080 1.01 12,167 0.93* 
2 F 4,293. 1.95 11,704 1.93 
M 4,495, 1.96. 12,225 1,92 
3 F 4,250 2.92 11,641 2.90 
M 4,530 2.91 12,255 2.89 
4 F 4,274 3.88 11,608 3.89% 
M 4,640 3:87 12,346 — 3.86* 
5 F 4,521 4.84 11,440 4.85* 
M 4,660 4.83 12,399 4.82* 
6 F 4,266 5.83 12,114 5,.82* 
M 4,670 5.80 12,468 5.80" 
7 F 3,941 6.88* 11,790 6.87* 
M 4,212 6.83* 12,450 6.82* 
8 F 4,197 7.89* 11,849 7.86* 
M 4,254 7.80* 12,418 --7.81* 
9 F 4,217 8.85* 12,259 8.82* 
M 4346 8.78* 12,644  8.76* 


Mathematics PAT T-Tests 
Enrolled Gender 05-06 05-06 06-07 06-07 
Grade N Mean PAT N Mean PAT 
Score Score 
3 F 4,083 33.12* 10,790 30.60* 
M 4,240 


34.07* 11,537 31.54* 


6 F 4,054 36.64" 11,451 34.47* 
M 4335 37.60* 11.923 36.03" 

¢ Ff 4,166 32.44 11,337 30.80* 
M 4,162 32.35 11,742 31.36" 


* indicates a statistically significant difference between males and females. 
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Student Mobility 


When a student changes schools, he/she must learn to deal with a new physical, social and 
learning environment - new teachers, new classmates, possibly different sets of rules, different 
learning expectations - and may start at a different point in the curriculum than he/she left 
behind. It also takes some time for teachers to determine the student’s learning level, learning 
style, interaction skills, etc., and thus define the optimal program for his/her ongoing learning. 


These issues contribute to the findings, within the research literature, of a negative relationship 
between the number of times a student changes schools in a given period and his/her academic 
growth in that period. Other studies (Wasserman, 2001) of this relationship in an Alberta setting 
have supported these findings and suggest that additional research would be useful, to not only 
enrich the understanding of the relationship but to highlight any situations in which the negative 
impacts may have been mitigated by helpful strategies to support better transitions for students 
and schools. The GLA data provides such an opportunity for additional research. It permits an 
on-going analysis of the relationship between the number of school changes students have made 
and their current grade level of achievement, thus allowing for an assessment of the cumulative 
impact of mobility. 


Student mobility is captured by Alberta Education, once at the end of September and again in 
March, and compiled in the Student Information System (SIS). The Student Mobility Indicator 
(SMI) provides an indication of the number of times a student has changed schools since entry 
into the Alberta school system. The SMI is calculated by counting the number of different 
school registrations each student has up until the most recent calendar year. Students could be 
changing schools more frequently than is captured, thus the numbers shown are a conservative 
estimate of student mobility. All students start with an SMI of | as they have all been registered 
in at least one school. Student mobility is then broken down into two categories — high and low. 
In Grades 1-3, high mobility students are those having a mobility indicator of 2 or more. 
Students having a mobility indicator of 1 are considered low mobility. In Grades 4-6, high 
mobility students are those having a mobility indicator of 3 or more. Students having a mobility 
indicator of 2 or less are considered low mobility. In Grades 7-9, high mobility students are 
those having a mobility indicator of 4 or more. Low mobility students have a mobility indicator 
of 3 or less. In the following tables the two categories of mobility includes students on a graded 
curriculum in Grades 1 to 9. 


Table 20 - Student Mobility, Mathematics, Provincial 


Mobility - Provincial Mathematics 


Percent of total. 


nF : studer tied. Tt nis enrolled (Ya) 
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Table 21 - Student Mobility, English Language Arts, Provincial 


Mobility - Provincial English Langua 


ge Arts 


Table 22 - Student Mobility, French Language Arts, Provincial 


Mobility - Provincial 


French Language Arts 


‘ : > ber: | Percent of total umh g Percent of total 


gd ’ enrolled (%) | students | enrolled (%) 


GLA equal to enrolled grade 


GLA above enrolled grade 


GLA NA 


The majority of students can be described as having low mobility (69.1% of all students with 
GLA data). One of the most salient observations from the above tables is that a greater 
proportion of high mobility students have a GLA below their enrolled grade level compared to 


low mobility students. This observation supports the hypothesis that mobility negatively affects 
student achievement. 


As noted above, student mobility was calculated based on the number of different school 
registrations a student had accumulated. Each student begins with an SMI of 1, indicating that 
they have been registered in one school. When a student registers in a different school, the SMI 


will increase by 1. The chart below lists the student mobility indicator values and the number of 
students having each SMI. 
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Student Mobility Indicator (SMI) 


Indicator — Frequency 


15-21 
Total 


* 15 students are missing a student mobility indicator 
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The chart above shows approximately 86% of students in our sample had 3 or fewer different 
school registrations. Bear in mind students in the table above are in Grades | through 9 and 
those students in the younger grades are the reason for the large number of students with a 
mobility indicator of 3 or less. The following chart displays SMI by GLA above or equal to 
enrolled grade as well as by GLA below enrolled grade for English Language Arts. 


Table 24 - Student Mobility Compared to GLA levels in English Language Arts 


Students’ GLA levels compared to Mobility Indicators in English Lan 


Mobility | GLA Equal or | 
Indicator 


Above 


suage Arts 

| 81,988 (85.5%) | 9.255 (9.7%) | 4,647 (4.8%) | 95,890 | 
| | 3,091 (75.8%) | 827 (20.3%) | 162 (4.0%) | 4,080 | 
a aa : 


7 
812 (67.7%) 
185,681 (85.5% 


* 15 students are missing a student mobility indicator 


The following two charts further examine the effect of student mobility on grade level of 
achievement in English Language Arts. 


Table 25 - Student Mobility and English Language Arts by Enrolled Grade 


ge 


= 


8.4% 


The next graph shows the mean GLA values in English Language Arts for each grade broken 
down by the student mobility indicator. 


Mean GLA per enrolled grade as SMI increases in English Language Arts 
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| 
| 
| 
| 
| 
| 
| 
| 


* Categories with fewer than 20 students have been removed 
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In Grades | through 6 the majority of students have an SMI of | indicating that these students 
have not changed schools. In Grades 7 to 9 the majority of students have an SMI of 2 illustrating 
the usual pattern of moving to a new school for junior high. 


Table 26 - Student Mobility Compared to GLA levels in Mathematics 


Students’ GLA levels compared to Mobility Indicators in Mathematics 


Mobility GLA Equal or GLA Below ~GLANA Count 
Indicator Above 


| __93,921 (89.7%) | 5,247 (8.7%) | 973 (1.6%) | ———_—60,141 | 
7 
10-21 
Total 21,344 (9.8%) 


* 15 students are missing a student mobility indicator 


Table 27 - Student Mobility and Math by Enrolled Grade 


Math- Student Mobili 


h 


ce 
° 
: 


238 (1.3%) | 18,188 (76.3% 
96 (1.3%) | __7,218 (29.4% 


573 (3.3%) | 17,231 (69.2% 
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The mean GLA values for Math in each grade were plotted against SMI values to produce the 
following chart. Once again, aside from the first grade there is a gradual decrease in the mean 
GLA as the SMI increases. 


Mean GLA per enrolled grade as SMI increases in Mathematics 


OorNWHU DH ~)1 C1 WO 


* Categories with less than 20 students have been removed 


The following graph illustrates the effect mobility can have on achievement. 


———— — 


Percent of Students Below Grade Level by Mobility Category for each 
Grade 


25.0% 
20.0% 
15.0% 
10.0% 
5.0% | 
0.0% 


Percent Below Grade 
Level 


As the graph above illustrates, there is a noticeable effect between high and low mobility. There 
is a statistically significant difference between the mobility groups in Grades 4 through 9. This 
significance may indicate that mobility has a more pronounced effect as students age. It may also 
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be due to the likelihood of students having a higher mobility indicator as their enrolled grade 
increases. 


Correlations between SMI and GLA 

Kendall’s tau-b values were calculated in order to illustrate the relationship between SMI and 
GLA groupings. Kendall’s tau-b is an alternative non-parametric form of rank correlations. The 
tau-b is used as an inferential statistic to show the strength of those relationships. Tau-b is used 
to measure the association between student mobility and GLA. This particular test was chosen 
as it uses ordinal level data based on pair by pair comparisons. The chart below details the 
correlations. 


Table 28 - SMI by GLA, Tau-b Calculations 


SMI by GLA- Grade and 05-06 06-07 


Subject Tau-b Tau-b 
Eng. LA and SMI 0.096* 0.103* 


Math and SMI 0.126* 0.117* 


* correlation is significant at the 0.01 level. 


The above correlations between student mobility and GLA are significant at the 0.01 level in 
both years. They are relatively weak in strength, although this is to be expected as student 
mobility can affect a student’s GLA but is only one of many determining variables. The 
correlations are similar between the two years of data indicating the validity of the data. 


Students not on a Graded Curriculum (Modified Programmin 


There were 3,380 students not on a graded curriculum in this study. The students were assessed 
based on the degree of achievement in their Individual Program Plans (IPP’s) relative to their 
foundational skills, academic readiness skills and life skills. The results reported below should be 
considered preliminary as data will become more meaningful as multi-year trends become 
available. 


Table 29 - IPP Foundation Skills 


Foundation Skills 


i 
Students Total Enrolled 

% 
| None ofthe skillsattained | | 3.3% | 
LZ ss x7 
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Table 30 - IPP Academic Readiness Skills 


Academic Readiness Skills 


Al clad abe 
Students | Total Enrolled 


Table 31 - IPP Life Skills 
ad Number of Percent of 
Students —_| Total Enrolled 
: oe) 


[None ofthe skills atained | _——_—*s0| 2.4% | 


Life Skills 


GLA and Enrolled Grade 
The following tables and graphs show GLA values by enrolled grade for all students, in English 
Language Arts and Math. N1 indicates students that are not yet at the Grade | level. 


Grade Level of Achievement by Enrolled Grade 


Grade One 


% of Total ‘eas % of Total 
Level 


2.320 
|1 sf 18,962 [80.0% | 21649 | 91.4% | 
_—- 
| Missing _| 


0%] It _0.0% | 
Missing 
| Total | 23,692 | 100.0% | 23692 | 100.0% | 
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GLA Value 
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Grade Two 


Number of students 


Grade Grade 3 
Level 


Number of students 


2 
3 
5 
7 


GLA Value 


Grade Grade 4 
Level 

| 49 | 
| 560 | 
| 2,101 | 

20,529 

| 84 | 
pe oly 
| 23,954 | 


Number of students 


Total 
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Grade Five 


Grade 


[NI 
‘ae 
ed 
[8 
ee! 
Missing | 
| Total__| 


Total 


P| 
4 
ee 
) ae 
9 | 
[10 
| Missing | 


ELA 


% of Total 


Number of students Number of students 


Number of students 


GLA Value 
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Grade 8 


N1123 4567891011 
GLA Value 


Number of students 
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Using the student counts from the above tables, the number of students whose GLA is below 
grade level in each grade are shown in the graph below. From Grade | to 6 there are a higher 
number of students attaining below grade level in English Language Arts. In Grade 7 to 9 there 
are a higher number of students attaining below grade level in Mathematics. 
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Number of Students Below Grade Level by Enrolled Grade 


Number of Students Below Grade Level 
by Enrolled Grade 
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Correlations between GLA and Enrolled Grade by Sub-Groups of the Population 
Correlations between the students’ GLAs and enrolled grades were calculated using Spearman’s 
tho to determine the “goodness of fit” between GLA and enrolled grade related to students’ 
being “coded” or not. The correlation between the two variables reflects the degree to which 
they “move” together. In this case, a high positive correlation coefficient results when an 
increase in enrolled grade is mirrored by the same increase in grade level of achievement. As 
was expected, GLA was highly correlated to enrolled grade, meaning the enrolled grade of a 
student typically matches their GLA. In English Language Arts this relationship was strongest 
for the gifted students, while students with severe disabilities had the lowest correlation between 
GLA and enrolled grade. In Math, the relationship was strongest for the non-coded students 
while students with severe disabilities had the lowest correlation. A comparison to the 2005-06 
GLA correlations is shown in the tables below. 


Table 32 - Correlations between GLA and Enrolled Grade, ELA 


English GLA 


2005-06 Correlation 2006-07 Correlation 
Student Codes Coefficient Coefficient 

Non-Coded 991 ** .991** 
Severe Disabilities .853** .868** 
Mild/Moderate Disabilities .845** 896** 
Gifted .994** .994** 
ESL Canadian Born (303) .974** ' .966** 
ESL Foreign Bom (301) .925** .945** 


**Correlation is significant at the 0.01 level (2-tailed) 
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Table 33 - Correlations between GLA and Enrolled Grade, Math 


Mathematics 
2005-06 Correlation 2006-07 Correlation 
Student Codes Coefficient Coefficient 
Non-Coded. - Niles “Egy, “Agog ee 
Severe Disabilities 863** 874** 
Mild/Moderate Disabilities 4), .854**_ _ mole" x 
Gifted .980** -984** 
ESL Canadian Born (303) ‘uh. ygg.985t¢ 975** 
ESL Foreign Born (301) 974% IT1%* 


** Correlation is significant at the 0.01 level (2-tailed). 


The graphs (next pages) show GLA by enrolled grade for all students as well as sub-populations 
of students, in English Language Arts and Math. The mean GLA value is plotted against the 
enrolled grade to show graphically the degree to which students’ GLA reflects or departs from 
their enrolled grade relative to sub-group affiliations. | As anticipated, the first two graphs 
demonstrate the close relationship between enrolled grade and GLA for non-coded students. 
This pattern is parallel to the high percentage of students who attain the acceptable standard on 
provincial achievement tests. 


Non-Coded Students — English Language Arts 


i Mean English LA GLA by Enrolled Grade - Non-coded Students | 


| 
| 


Enrolled Grade 


Mean GLA 


| 
| 
4 Enrolled Grade | | 2 
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Non-Coded Students ~ Mathematics 


ln - .  — . ae  e e f n,  e i  e] 
Mean Math GLA by Enrolled Grade - Non Coded Students | 


7 | |—e— Enrolied Grade 
} | —#— Mean GLA 


Mean GLA 


Students with Severe Disabilities- English Language Arts 


Mean English LA GLA by Enrolled Grade - Severe Disabilities | 


—e— Enrolled Grade! 


~—#— Mean GLA | 


Mean GLA 
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Students with Severe Disabilities — Mathematics 


| Mean Math GLA by Enrolled Grade - Severe Disabilities 


Mean GLA 


1 2 3 4 5 6 7 8 9 
Enrolled Grade 


Students with Mild/Moderate Disabilities — English Language Arts 


Mean English LA GLA by Enrolled Grade - Mild/Moderate Disabilities 


: 
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Students with Mild/Moderate Disabilities — Mathematics 


| Mean Math GLA by Enrolled Grade - Mild/Moderate Disabilities 
| 10 
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Gifted Students — English Language Arts 


Mean English LA GLA by Enrolled Grade - Gifted 
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Gifted Students — Mathematics 


Mean Math GLA by Enrolled Grade - Gifted 
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ESL Canadian-Born Students — Mathematics 


Mean Math GLA by Enrolled Grade - ES L Canadian Born (303) 
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ESL Foreign-Born Students — Mathematics 


Mean Math GLA by Enrolled Grade - ESL Foreign Born (301) | 


>| | —e— Enrolled Grade 
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Discussion 
The graphs above yield some interesting observations: 


The relationship between enrolled grade and GLA for non-coded students is very close as 
was expected. 

The variance between enrolled grade and GLA tends to increase through the elementary 
grades and peaks at grade 6 and grade 9 for students with a severe disability. 

A similar pattern is seen for students with a mild/moderate disability, but the peak in grade 
nine is absent. 

Gifted students are more likely to have a GLA above enrolled grade in Mathematics than in 
Language Arts. 

Foreign-born and Canadian-born ESL students’ GLA distributions are relatively similar. 
Foreign-born ESL students seem to have a GLA on par with their enrolled grade in 
mathematics up until Grade 5. In English Language Arts the two variables diverge as early 
as Grade 2. 
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Birth Month-Combined Grades 


The following table shows the breakdown of student birth month. When grade levels are 
combined there is a relatively equal distribution of students across birth months, but with fewer 
students in the second January and February categories. 


Table 34 — Student Birth Month 


Percentage 
1* Janua 5.7% 
1" Februa 6.1% 
March 2.26 
Pe 


ae 17,778 1% 
1% 


73% 
ie : 9% 
<r ust : = 

September 
Ao a 
November 6.6% 
December 5.6% 
ie Janua 3.3% 

"! Februa 2.3% 
on 100.0% 


*Not all students were included i in this table as some students were outside the parameters of analysis meaning they were much 
older than the students being examined in this table. 


The table above illustrates the distribution of students by birth month for all grades of students. 
The distribution was calculated by examining each grade individually and then combining all 
grades. Students in the 1“ January and February are usually the oldest students in each grade, 
whereas students in the 2" January and February are the youngest in each grade. 


GLA and PAT by Age within Grade Cohorts 

Previous Alberta Education studies have indicated that there is a relative age effect between 
average PAT scores and birth month within grade cohorts, where older students tend to have 
higher average test scores than the younger students when measured by the z-score of average 
PAT results for each birth month group (Alberta Learning, 2001). 


A comparative analysis was undertaken using GLA data. The percentages of students ‘at or 
above’ their grade level in English Language Arts and Math were converted to z-scores® and 
plotted. (See graphs below). There is a hee age effect Grades | through 5, which is most 
pronounced in Grade 1. In Grade 1, 1“ January to October are statistically significant when 
compared to 2™ January and 2™ ike 


* Z-scores or standard scores convert a distribution of a set of scores to a normal distribution that allows a more 
consistent analysis of the data, A z-score of +1.0 represents a score equal to one standard deviation above the mean. 
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The following graphs were produced using the z-scores for Grades 1 through 5, English 
Language Arts GLAs for students in the GLA dataset. 


Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade | students 


Ist March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Feb Jan Feb 


a z-score -—— Linear (z-s score) | 


Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade 2 students 


RRA Soo——nw 
SSS8SS8SS838 


Ist March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Feb Jan Feb 


[_s= 2score — Linear score) 
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Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade 3 students 


Ist 1st March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Jan Feb Jan Feb 


—*— z-score ——— Linear (z-score) 


The graph below demonstrates that when PAT scores are recoded into “percent at or above 
acceptable” to mimic the GLA data, the relative age effect remains, In Grade 3, March and April 
are statistically significant when compared to 2" January. 


Z-score of the percent of students at acceptable or above on the Language Arts 
PAT, by birth month with cohort - Grade 3 students 


a 
_ z-score =—— Linear (z-score) | 
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Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade 4 students 


ie es lee = S 


Ist March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Feb Jan Feb 


— 


Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade 5 students 


¢ = ae 7 y = : $ 3 


Ist 1st March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Jan Feb Jan Feb 


| —*— z-score ———= Linear (z-score) | 


— —— 


In Grade 5, May is statistically significant when compared to August, September and October. 


Discussion 
The age effect, which illustrates that younger students achieve at a lower level than those 


students born earlier in the year, is apparent in English Language Arts in Grades | through 5. 
After Grade 5, the age effect tapers off and is no longer apparent. In this sample the age effect 
persisted longer than in previous analyses of GLA data where the effect dissipated by Grade 3. 
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GLA at a Glance 


The following table shows all groups of students and their GLA in Mathematics and English 
Language Arts including students for whom GLA data was either not available or not applicable. 


Table 35 - GLA Overall Summary Table, Including ‘GLA NA’ 


Overall Summary Table English Language Arts 


Student Codes Frequency % Ator | % Below | %GLA| %Ator | % Below] % GLA 
Above NA Above 


NA 
224%) 85.4%] 11.1%] 3,5%) 
90.6% 
- Emotional/Behavioural 58.8% 35.9% 
Disabilit 
|- Deafness | CS | 46.2%] 46.2%] 7.7%] 35.9% —dSALT% _—_—O92'.AMG 
Disabili 
Disabili 
96.0% 
Pee POISE: GATE GE Ge QE Fe eee 
88.1% 
"SS JAG GE) WE eee RE ees ee 
| Student Mobility ss [| CT tC“‘R;SSOOCOCOC;*dSC (N$R’NNN C— <@s—‘(sSdT|sC CdTCCidCY 
ES a 
88. 


category to be shown separately. 
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Note: The categories of ‘at grade level’ and ‘above grade level’ have been combined. There are not enough cases in the ‘above grade level’ 


The following table shows all groups of students and their GLA in Mathematics and English 
Language Arts excluding students for whom GLA data was either not available or not applicable. 
In this table asterisks indicate significant differences between a specific cohort of students and 
the non-coded cohort. This analysis is important as it points to factors that may be limiting 
academic success for cohorts of students who are most at risk of under-achievement and thus 
would benefit from more focused systemic responsiveness to their learning needs. 


Table 36 - GLA Overall Summary Table, Excluding ‘GLA NA’ 
English Language Arts 


Student Codes % Ator % Below | Frequency | % Ator 
Above Above 


209,804 
3,28 97.6%* 
11,05 


8 
82. 

10,12 79.6%* 

f | 

107,412 86.2%* 

102,392 

ee | 

56,220 

0. 


* 
153,579| _90.4%* 
Birth Month rsd 


Note: The categories of ‘at grade level’ and ‘above grade level’ have been combined. There are not enough cases in the ‘above grade level’ 
category to be shown separately. 

*Denotes a significant difference at p<0.001 when compared to the non-coded sample of students. 

Note; Statistical testing was only done with the larger categories, disability types (severe and mild/moderate) were not tested against the 
non-coded sample, 
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Comparison of GLA and PAT data 
The GLA by PAT analysis demonstrates that GLA data can indeed supplement PAT data with 
reasonable reliability and validity. 


Correlations between PAT and GLA 

Kendall’s tau-b was used to measure the association between PAT and GLA. This particular test 
was chosen as it uses ordinal level data based on pair by pair comparisons. The chart below 
details the correlations for 2005-06 and 2006-07. 


The PAT and GLA variables were re-coded into the dichotomous categories: either ‘Below 
Acceptable’, or “At or Above Acceptable’ for PATs; and either ‘Below Grade Level’ or ‘At or 
Above Grade Level’ for GLA, and then the two dichotomous variables were compared. All 
relationships tested were at the p <.01 levels meaning they were significant. The p-value shows 
that the observed relationships are not due to chance. The following tables shows all tau-b 
values for the relationships tested and from this, one can conclude that the relationships are 
moderate in strength. A perfect relationship of 1.0 between GLA and PAT is neither an expected 
nor a desirable condition given the inherent differences in the evaluation designs which would 
underlie potentially different learning outcomes being measured with different assessment 
methods. 


Table 37 - PAT by GLA Tau-b Calculations 


2005-06 2006-07 
PAT by GLA- Grade and Subject Tau-b PAT by GLA- Grade and Subject Tau-b 
Gr. 3 Eng. LA 378 Gr. 3 Eng. LA .324 
Gr. 6 Eng. LA 406 Gr. 6 Eng. LA 337 
Gr. 9 Eng. LA 338 Gr. 9 Eng. LA 323 
Gr. 3 Math 388 Gr. 3 Math 342 
Gr. 6 Math - 403 Gr. 6 Math 366 


Gr. 9 Math 399 Gr. 9 Math 409 


When comparing 2005-06 to 2006-07, the strength of the relationships has decreased. This 
decrease in strength may be attributed to the larger sample size in 2006-07 and therefore may be 
a truer picture than the correlations of the previous year. The only exception is the increase in 
strength of the relationship between the Grade 9 PAT and GLA results. 
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GLA by PAT Analysis-Comparisons Using Achievement Levels 


In order to further examine the relationship between the GLA data and provincial achievement 
tests (PATs) and provide an additional perspective on these relationships, both PAT and GLA 
data were again re-coded into the categories of ‘Below Grade Level and GLA NA’ and ‘At or 
Above Grade Level’ for GLA; and ‘Acceptable or Excellence’ and ‘Below Acceptable and 
Excused or Absent’ for PATs. These groupings were chosen based on the current Alberta 
Education standard for cohort reporting. The groups were then cross-tabulated with the 
hypothesis being that students who score at or above the acceptable level on PATs tend to be at 
or above grade level, and likewise those that score below acceptable tend to be below grade 
level. The following tables show some support for the hypothesis, as 76-78% of the students in 
Language Arts and 65-80% of the students in Math who are at grade level are also at or above 
the acceptable level on the PATs. The data in the following tables includes all students from 
schools that submitted GLA data for 2006-07 in Grades 3, 6 or 9. 


Table 38 - Comparison of English Language Arts PAT and GLA 


Comparison of English Language Arts PAT and GLA 


Grade Level of Achievement — English Language Arts 
At or Above Grade — Below Grade Level Total 


Level or GLA NA 

PAT - Grade 3 Accept. or Excellence 77.6% 5.7% 83.4% 

English Language (17,304) (1,276) (18,580) 

Arts Below Accept., Excused 8.1% 8.5% 16.6% 
orAbsent (1,801) — (1,905) (3,706) 

Total 85.7% 14.3% 100.0% 

(19,105) (3,181) (22,286) 

PAT - Grade 6 Accept. or Excellence 77.9% 5.4% 83.3% 

English Language (18,173) (1.266) (19,439) 

Arts Below Accept., Excused 8.1% 8.6% 16.7% 

or Absent (1,887) (2,014) (3,901) 

Total 85.9% 14.1% 100.0% 

(20,060) (3,280) (23,340) 

PAT — Grade 9 Accept. or Excellence 76.2% 4.3% 80.5% 

English Language 17,489 982 18,471) 

Arts Below Accept., Excused 10.8% 8.7% 19.5% 

or Absent (2,483) (1,999) (4,482) 

Total 87.0% 13.0% 100.0% 

19,972 2,981 22,953) 


Note: All of the above observed relationships were significant when measured by Chi square. Bolded numbers 
represent inconsistent relationships between GLA and PAT data. 
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Table 39 - Comparison of Mathematics PAT and GLA 
Comparison of Mathematics PAT and GLA 


Grade Level of Achievement — Mathematics 
Ator Above Grade — Below Grade Level Total 


Level or GLA NA 

PAT - Grade 3 Accept. or Excellence 79.6% 3.5% 83.2% 
Mathematics (17,753) (788) (18,541) 
Below Accept., Excused 9.5% 7.3% 16.8% 

ioeitbsent * Put Pie gx d* i" 5 422g) OF OM C63) a ol 3, 75) 

Total 89.1% 10.9% 100.0% 
(19,873) (2.420) (22,293) 
PAT - Grade 6 Accept. or Excellence 74.0% 3.0% 77.0% 
Mathematics (17,279) (701) (17,980) 
Below Accept., Excused 13.4% 9.5% 23.0% 
or Absent (3,134) (2,226) (5,360) 
Total 87.5% 12.5% 100.0% 
(20,413) (2.927) (23,340) 
PAT — Grade 9 Accept. or Excellence 65.8% 3.0% 68.7% 
Mathematics (15,056 (679 15,735) 
Below Accept., Excused 18.0% 13.2% 31.2% 

or Absent 4,129 3,022) 7,151 
Total 83.8% 16.2% 100.0% 

19,185 (3,701 22,886 


Note: All of the above observed relationships were significant when measured by Chi square. Bolded numbers 
represent inconsistent relationships between GLA and PAT data. 


It should be noted, in reviewing the above two tables, that more students are categorized as 
‘below grade level’ in the PAT results than is true in GLA ratings. To illustrate, using the 
Mathematics table above, the following comparisons can be made: 


Table 40 - Students below acceptable, excused or absent on the Mathematics PAT 


Test Total # of # Students At or Above | Percent of Students At or Above 
ieee Students GLA Grade level GLA Grade Level 
PAT Math 3 
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Table 41 - Students below grade level or GLA NA on GLA Mathematics 


Test Total # of | # Students Acceptable or | Percent of Students Acceptable 
Students Excellent on PAT or Excellent on PAT 
PAT Math 3 2.420 32.6% 
7 


PAT Math 6 
| PAT Math 9 


The following tables illustrate students below grade level on GLA and below acceptable on PAT 
in English Language Arts. 


Table 42 - Students below acceptable, excused or absent on the English Language Arts PAT 


Test Total # of # Students At or Above | Percent of Students At or Above 
Students GLA Grade level GLA Grade Level 
PAT ELA 3 


bdoO TEE FTI 
PAT ELA 6 3,901 48.4% 
PAT ELA 9 4,482 2,483 ' 55.4% 


Test Total #of | # Students Acceptable or | Percent of Students Acceptable 
Students Excellent on PAT or Excellent on PAT 


PAT ELA 9 


It would appear from the above tables that while nearly half of the students who do not attain the 
acceptable standard on the PAT are considered at or above grade level, a smaller percentage of 
those who are considered below grade level are able to attain the acceptable standard on the 
PAT. This suggests that in terms of evaluating acceptable progress, the PAT is a more difficult 
standard to attain than is the GLA. The two assessments can be expected to demonstrate some 
variance because they are different forms of assessment, but both are designed to assess whether 
a student has met grade level standards. Therefore, one might expect that the variability would 
be seen as much in one direction as the other. The fact that this is not true suggests several 
possibilitics: 1) that it may be more difficult for teachers to assign a ‘below grade level’ 
evaluation to one of their students than is the case for markers of the PAT assessments, 2) 
student performance on PATs may be attenuated by test anxiety; 3) students may perform better 
on many assessments over time than on a single paper and pencil test, or 4) a combination of 
these factors. 


32.9% 
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Analysis of Students Below Grade Level 


In this section the ratings given by teachers through the GLA are further compared to PAT 
results in Grades 3, 6 and 9, In each case, it is possible to identify the students who are rated as 
below grade level by their teachers (GLA) and those rated as below the acceptable standard by 
the PAT. 


According to classical test theory (Crocker and Algina, 1986) all assessments are subject to some 
degree of test error. Given this fact the more achievement data available the clearer the true 
picture of a student’s achievement becomes. One would expect some differences as well as 
complementary relationships in the designation of individuals in the two ratings provided by 
PAT and GLA data. Classroom assessment is based on an array of assessment methods over 
time for the GLA rating, ideally measuring the full and complete range of learning outcomes; 
whereas the PAT is a single, highly valid and reliable, selected and constructed response test that 
typically measures between 66% and 95% of the curricular outcomes in the tested subjects’. The 
PAT’s are very likely the single best criterion-referenced assessment instrument available to the 
classroom teacher. Since the objective of both methods is to measure and provide evidence on 
how well a student is achieving as compared to the learning outcomes in the program of studies, 
one would expect a generally positive relationship between the students identified as “below” by 
both methods, and that is what we see in this data sct. Where anomalous or inconsistent 
relationships are observed it presents the occasion to ask why and to delve deeper into the data to 
help understand why the measures differ. 


An examination of the GLA pilot data for English Language Arts and Mathematics shows that 
this assumption departs most dramatically for Math 9. 


* These figures were obtained from a discussion paper presented to the Program and Accountability Advisory 
Committee, November 8, 2007. 
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Note: The data presented in the above tables are discrete data points. The lines were included to emphasize the patterns of the 
data. 


a Students Below Grade Level or Below Acceptable on PAT in 
| Mathematics 


35% pe 
30% + 
25% bi. 
20% 
15% + 


| 
| 
| 
) 7 
| 
| 
| 


¥ 2 


Note: The data presented in the above tables are discrete data points. The lines were included to emphasize the patterns of the 
data. 


Discussion 

A primary reason for provincial aggregation of Grade Level of Achievement data is evaluation of 
education programs such as special education, English as a Second Language, etc. This 
observation is particularly relevant for those grades that do not have PAT testing where GLA can 
provide data that would otherwise not be available. Additionally, it is useful to be able to 
supplement PAT data with GLA data in Grades 3, 6 and 9 as the added advantage would be 
broader and richer data to inform program evaluation related decisions. An example is the 
analysis of students below grade level or the analysis of students missing PAT data found on the 
next few pages. 


Further, the fact that the tau-b values show moderate strength lends credibility to the process of 
collecting GLA. A perfect relationship of 1.0 between GLA and PAT is neither an expected nor 
a desirable condition given the inherent differences underlying the evaluation designs. PAT data 
are derived from a single paper and pencil test whereas GLA data are based on numerous and 
more dynamic observations over time, and thus should be a much richer method of assessment, 
which one could reasonably assume to produce, positively correlated albeit different data than a 
PAT result. Considered together, PAT and GLA data provide a richer and more comprehensive 
picture of student achievement. 
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GLA for Students Missing PAT Data 


In order to determine whether GLA provides useful data for students for whom PAT data was 
unavailable, GLA data for these students was analyzed. Relative to the Grade 3, 6 and 9 students 
included in the 2006-07 GLA data set, the following table defines the breakdown of student 
categories and percent of students lacking PAT results and the corresponding GLA results. 


Table 44 - Student Missing PAT Data Grade 3 


Student Code Groups 


Non Severe Mild/Moderate | Gifted ESL - ESL - All 
Coded Cdn-born ee Students 


Grade 3 Total Eligible 17,405 605 1,398 151 1,642 1,320 22,376 
Englis h to write PAT 


% of students 96.1% 64.1% 72.5% 98.7% 81.7% 93. 0% 


writing PAT 

PAT 

a Ss 
Level 


Language 
Arts 
Students 
Grade 3 
students 
with no 
PAT data 


At Grade Level 
or Above 


No GLA | 
Bi! [a Cane oe, 23. | ee | : 
GLA 


% of students Hise A% Five! 7% ee 5% Bie 0% Me! 1% 60. hee eae! 6% 
with no PAT 
data having 
info using 
GLA 
info using GLA, PAT or both 


Grade 3 English Language Arts GLA data provided data for 82% of the population for whom no 
PAT data were previously available. Further, when GLA data are substituted, data are 
unavailable for only 5.1% of the sub-population coded severe (compared to 35.9% missing PAT 
data). Similarly, using GLA means data are unavailable for only 1.8% of the students in the 
dataset coded mild/moderate (compared to 27.5% using PAT data). Similar findings for Grades 
6 and 9 in English Language Arts are summarized below. 
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Table 45 - Students Missing PAT Data Grade 6 
Student Code Groups 


Non Severe Mild/Moderate | Gifted ESL - ESL - All 

Coded Cdn-bom a Students 
Grade 6 Total Eligible 17,828 2,287 1,167 23 . 
Englis h to write PAT 


% of students 95.8% 54.5% 73.9% 99.2% 91.6% 86.1% Bias! 2% 
writing PAT 
PAT 
i 1 
Level 
a 
or Above 


[NoGLA | ae) a | ee 


17. ia 3.7% — 14.3% 28.7% ; 3% 
GLA 


% of students Bed 9% 90.3% 96.3% 100.0% 85.7% 71.3% Miecal 5% 
with no PAT 
data having 
info using 
GLA 
using GLA, PAT or both 


Table 46 - Students Missing PAT Data Grade 9 


Language Arts, 
Students 
Grade 6 
students with 
no PAT data 


Student Code Groups 


Non Severe Mild/Moderate | Gifted ESL — ESL- All 
Coded Cdn-born Foreign- Students 
Ta 


Grade 9 Total Eligible 18,677 ane 113 
Eng lish to write PAT 


% of students 94.5% | 46.5% 69.6% cae 1% horse 9% 


writing PAT 
PAT 


Language Arts 
Students 
Grade 9 


students.with” |p iow Grade 225 
no PAT data Level . 


At Grade Level ala 132 15 42 839 
or Above 
[NoGLA | ssa] 4H TCC STS 


25 Md Baad a 3% Bad 0% 9. Baal 30. iad a 8% 
GLA 


% of students — 0% 81 Bad al 7% in 0% ia! 2% 69. Mase Beal 2% 
with no PAT 

data having 

info using 

GLA 


% of Grade 9 students having info 93.0% 100.0% 97.8% 
using GLA, PAT or both 
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As the charts above demonstrate, GLA provides data for the majority of students who would 
otherwise not have any data provided. 


Analysis of Grade ELA Introduced for FLA Students 


In the 2006-07 GLA pilot, schools were asked to enter the grade English Language Arts was 
introduced to French students. The following table shows the distribution of the grade in which 
ELA was introduced. 


Table 47 - Grade ELA Introduced Distribution 


Grade ELA Intro Frequency Percent 


I 1,47] 17.0% 
2 2,495 28.8% 
3 4,535 52.3% 
4 34 0.4% 
5 1] 0.1% 
6 14 0.2% 
7 86 1.0% 
8 19 0.2% 
9 8 0.1% 
Total 8,673* 100.0% 


*There were 4,013 students who did not have Grade ELA Introduced information 


The majority of students had English Language Arts introduced in Grades 1 through 3. The 
following tables illustrate the French and English Language Arts GLA compared to the grade in 
which ELA was introduced. The tables only contain those students who had ELA introduced in 
Grades | through 3 as they are the largest group of students. 


Table 48 - Grade ELA Introduced compared to French Language Arts GLA 


Intro Above ei 
Pp 1277 (86.8%) | 194.(13.2%) | 00.0%) | i471 


Note: All of the above observed relationships were significant when measured by Chi square. 


Table 49 - Grade ELA Introduced compared to English Language Arts GLA 


Grade ELA GLA Equal or GLA Below —§| GLANA Count 
Intro Above 


‘eed ‘sit 1,271 (86.4% 200 (13.6% 0 (0.0% 1,471 
2.434 (97.6%) 61 (2.4%) 0 (0.0% 
4,358 (96.1% 176 (3.9% 1 (0.0%) 


Note: All of the above observed relationships were significant when measured by Chi square. 
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As the above tables illustrate, there are statistically significant relationships between the grade 
ELA introduced categories. When examining the GLA results for both FLA and ELA, students 
with an ELA introduced of Grade 2 seem to be the largest proportion of students attaining a GLA 
equal or above their enrolled grade. 


Due to the variance of the grade in which ELA is introduced as shown in the above frequency 
table, it was decided that this field would continue to be reported in future GLA data submissions 
for French students. 


Analysis of Students with Multiple Years of GLA Data 


There were 64,170 students involved in both the 2005-06 and 2006-07 GLA data collections. 
There were 82,390 student records submitted in 2005-06 and 220,682 in 2006-07 with 78% of 
the 2005-06 student cohort having data for both years. The following analysis is based only on 
those students for whom GLA data is available for both years and is presented only to illustrate 
the type of trend analysis that will become viable as the GLA initiative becomes implemented 
over several years. 


Students enrolled grade in 2005-06 and 2006-07 are shown below. 


Enrolled Grade 


2005-06 2006-07 
Enrolled Frequency Percent Frequency Percent 
Grade 
] 7,929 12.4% 104 2% 
2 8,234 12.8% 7,892 12.3% 
3 8,210 12.8% 8,200 12.8% 
4 8,225 12.8% 8.212 12.8% 
5 8,424 13.1% 8,227 12.8% 
6 7,535 11.7% 8,404 13.1% 
a 7,684 12.0% 7,558 11.8% 
8 7,898 12.3% 7,665 11.9% 
9 31 0% 7,908 12.3% 
Total 64,170 100.0% 64,170 100.0% 


The following table shows the English Language Arts GLA codes for all students submitting in 
both years. 
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GLA above enrolled grade 
GLA NA 
64,170 100.0% 


*Indicates a significant difference between 2005-06 and 2006-07 


Percent of Students in Each of the ELA GLA Categories | 
el g® | 


el As 


90.0% 
80.0% + 
70.0% | 
60.0% += 
50.0% 
40.0% 4) 
30.0% gy 
20.0% +See 
10.0% 
0.0% + 


GLA below GlAequalto GLA above GLA NA | 
enrolled grade enrolled grade enrolled grade | 


There is a drop in the number of students attaining the above grade level category and there is an 


increase in the number of students attaining below grade level. In Mathematics the same patterns 
emerge. 


Table 51 — All Students, Mathematics 


All Students - Mathematics 
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*Indicates a significant difference between 2005-06 and 2006-07 
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GLA below GLAequalto GLA above GLA NA 
L enrolled grade enrolicd grade enrolled grade 


Comparison of English Language Arts, 2005-06 to 2006-07 


To better understand the student’s changes in grade level of achievement from 2005-06 to 2006- 


07, the following tables were produced. 


Table 52 — 2006-07 Results for Students At or Above Grade Level in 2005-06, 
English Language Arts 


Students At or Above Grade Level in 2005-06 - English Language Arts 


« 
# 


Total 55160 | 


oe 


— —— 
Pe Kos t 
Eh. 
4 s 
2 


ee 
Oo 


100.0% 


Table 53 — 2006-07 Results for Students Below Grade Level in 2005-06, English 


Language Arts 


Students Below Grade Level in 2005-06 - English Language Arts 


A a eS 
ieee gs 
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GLA below enrolled grade 


GLA equal to or above enrolled grade 


GLA NA 
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Comparison of Mathematics, 2005-06 to 2006-07 


Table 54 — 2006-07 Results for Students At or Above Grade Level in 2005-06, 
Mathematics 


PAPE Eee" nA 
“ee Number of | Percent oftotal 

ibais hi ial Jf students | ional %) 

GLA equal to or above enrolled grade 


Table 55 — 2006-07 Results for Students Below Grade Level in 2005-06, 
Mathematics 


Students Below Grade Level in 2005-06 - Mathematics 
Fe Ee 2006-07 


r oe re it a 


a ae 3s 


oes: rf sD = . ced 
GLA below enrolled grade 


From the above tables it is apparent that in Mathematics, most students who were at or above 
grade level in 2005-06 attained the same categorization in the following year (92.8%). For 
students categorized as below grade level in 2005-06, 31.8% achieved a GLA equal to or above 
their enrolled grade in the following year. A similar pattern is shown for English Language Arts. 


Change in GLA by Enrolled Grade 

The following tables show students’ GLA in English Language Arts and Mathematics for 2005- 
06 and 2006-07. In order to be part of this analysis, students had to have advanced an enrolled 
grade trom 05-06 to 06-07. Five hundred (500) students were excluded from this analysis. 


Grade One to Grade Two 
Grade 1 - 05-06 Grade 2 — 06-07 


Grade % of Total |-ELA | % of Total 


|__| Grade 1 - 05-06 Grade 2 — 06-07 


Grade ional % of Total | Math | % of Total 
Level 


[NI] 388{) SOT 87 
ja | 7227] | ot | 7.8 
[2 | 3 T7042 | 90.0 | 
Es Es a | ee 2 
Es a Se ee 
|Missing| 174] 2.1] 65 0.9 
7,823 | 100.0 | 7,823 | 100.0 | 


54 


Grade Two to Grade Three 
fs Grade 2 — 05-06 Grade 3 — 06-07 


Level 
17 
2 


a 
19 
36| 10.2 | 
9 


Cs a a a 
[2 | 6891] 84.5 | 836 | 10.2 
| 24 | 
5 


| 35] 4 | 6,913] 84.7 | 


yf 
a 0.3 
1.7 


a ee | 03 | 
5 aD ee Se ee 
issi 36] 130] 1.7 | 
| 8,159 | 100.0 | 


as Grade 3 — 05-06 Grade 4 - 06-07 


Grade % of Total | ELA | % of Total 
Level 


[NL | 68] OB] S| 02 | 
as eC) a a 
2] aot 84] tt 27 | 
[3 | 7,033] 86.2 [803] 98 | 
4 C403 | 6,963] 85.3 | 
a ae | a | 9 
6 | OC | 
| Missing | _139 | 1.7 0.7 
| Total | 8,163 | 100.0 | 8,163 | _100.0 | 


| rade 2 ~ 05-06 
~ do ane lt 
Level 

pNI | 93 | TG 

a | ee 

}2 | 7.466{ S| 84 | 8 

a es Ye | ee 

is | TC 


Total | 8159] 100.0 | 8,159 | 


ra Grade 4 — 05-06 Grade 5 — 06-07 


Level 


LNT | as] | 00 | 
pi | OTS | 


Cs es 2 
a es) a | 7 
ae 2 a) ee 
| Missing] 156] S20] 52] 0.6 | 
| Total | 8173] 100.0 | 8,173 | _—100.0 | 
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Grade 6 


12 

61 

7,24 
45 


05-06 


% of Total 


ayant 
olo|— 


% of Total 


Grade 5 
Grade 7 — 05-06 


‘oe % of Total heieinall % of Total 


129 


| Total | 8361| 100.0 { 8.361 | 100.0 | 


|e pel Grade 6 — 05-06 Grade 7 — 06-07 


as 
Level 
Level 


06-07 
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| __—100.0 | 
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Grade 7 — 06-07 
% of Total 


rade 8 — 06-07 
% of Total 


Grade 6 


< 
- 
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05-06 


% of Total 
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Grade Five to Grade Six 
Grade Six to Grade Seven 
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Grade % of Total | ELA % of Total 
Level 
| 7 
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The tables above illustrate interesting patterns of grade level achievement. For students who are 
evaluated as being below grade level, there are many that have a grade level of achievement that 
is more than one grade below their enrolled grade. It is interesting to note that in Mathematics, 
every group of students in the 2005-06 cohort, the percent of students at grade level decreases 


from 2005-06 to 2006-07. 


Conclusions 


This report described the processes and outcomes associated with the 2006-07 Grade Level of 
Achievement (GLA) pilot data collection, data management and data analysis. The key 
questions addressed by this report were 1) can GLA data be collected and reported on a 
consistent basis, and 2) does the data contribute to our knowledge base regarding student 
achievement? Results of this and previous studies of GLA pilots demonstrate that the answer to 
these two questions is yes and is demonstrated by the following key findings: 
e The degree of jurisdiction participation in the 2006-07 pilot was much higher than expected. 
A participation rate of 33 % was expected; however, the data submitted represents 60% of 


schools. 


¢ The error rate for data submission was low with only 2.2% of the files submitted having 


errors. 


e Compared to the 2005-06 data, the 2006-07 data demonstrates similar results for the total 
cohort and for specific sub-groupings of GLA data. Patterns of data distribution observed in 
2005-06 with 82,000 students were also observed in the 2006-07 data. 

e The data analysis demonstrates many interesting relationships. For example: 

o The large difference between Math 9 Provincial Achievement Test data and GLA 


continues to be evident. 


o Significant variations are observed in GLA between student sub-groups. The 
GLA at a Glance (Table 36) illustrates this. 

o A much higher percentage of students coded as gifted are assessed above grade 
level in Math than in English Language Arts or French Language Arts. 
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When comparing foreign-born and Canadian-born students, both groups of 
students perform similarly in Mathematics but in English Language Arts, 
Canadian-born students are at an advantage. 

Although a serendipitous finding, there is a disproportionate ratio of male: female 
severely coded students. There are nearly four times as many males as females 
coded as severely disabled. 

When comparing GLA and PAT data, females outperform males in GLA to a 
statistically significant degree in both subjects in nearly all enrolled grades. The 
reverse is seen on the Math PAT results with males outperforming females to a 
statistically significant degree. In English Language Arts, females are 
outperforming males on the PATs. 

A greater proportion of high mobility students (i.e. those who have changed 
schools more frequently than other students) have a GLA below their enrolled 
grade level. 


When examining mean GLA compared to enrolled grade, interesting patterns emerge: 


° 


The relationship between enrolled grade and GLA for non-coded students is very 
close as expected and is generally aligned with PAT results. 

The variance between enrolled grade and GLA tends to increase through the 
elementary grades and peaks at grade 6 and grade 9 for students with a severe 
disability. 

A similar pattern is seen for students with a mild/moderate disability, but the peak 
in grade nine is absent. 

Gifted students are more likely to have a GLA above enrolled grade in 
Mathematics than in Language Arts. 

Foreign-born and Canadian-born ESL students’ GLA distributions are. much 
closer together this year than was seen in the 2005-06 data. 

Foreign-born ESL students seem to have a GLA on par with their enrolled grade 
in Mathematics up until Grade 5. In English Language Arts the two measures 
diverge as early as Grade 2. 


The age effect'® is apparent in English Language Arts in Grades 1 through 5. After Grade 5, 
the age effect tapers off and is no longer apparent. This relationship is more pronounced in 
the 2006-07 GLA data than in 2005-06, possibly due to the larger number of students in the 


2006-07 data. 


There are moderate strength correlations between PATs and GLA (Grades 3, 6 and 9). This 
demonstrates a reasonable degree of concurrent validity of the GLA data. 


Using GLA and PAT information, GLA can provide important information for students that 


would not otherwise be available for students in grades not tested by PATs. 


Future Analysis 


As noted in the Limitations section of this report, given the pilot nature of the data presented, the 
findings must be considered preliminary and can provide only tentative conclusions regarding 
levels of student achievement and factors that affect achievement for specific student cohorts. 


'° Age effect is defined as older students in a grade tending to have higher average test scores than the younger students in that 


same grade when measured by the z-score of average PAT results for cach birth month group (Alberta Learning, 2001). 
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Nevertheless, this report points to a number of potential research questions that may require 

further analysis as GLA reporting becomes more confirmatory as full implementation is attained 

in the 2007-08 school year. Among the questions that may require investigation are: 

e Does the grade 9 PAT in mathematics or the GLA for mathematics 9 represent the best 
predictive validity for subsequent high school math achievement? 

e What are the reasons for classroom-based assessment providing higher marks for female vs. 
male students in mathematics contradictory to PAT marks? 

e Why are gifted students more likely to have a GLA above their enrolled grade in 
mathematics than in Language Arts? 

e Why is the proportion of male students coded with a severe disability so much higher than 
female students? 

e What strategies might be most effective in offsetting the negative effect of mobility on 
student achievement? 

e What pedagogical adaptations might be helpful in mitigating the age effect in the early 
grades? 


In addition, in future years the 2006 census data will be available for analysis. At that time, the 
census data will be compared to the 2007-08 GLA data to provide an indication of the extent to 
which variables external to the school influences student achievement. To create the most 
accurate comparison of the census data and the GLA data, the census data will be broken down 
by enrolled students’ postal code. 


The organization of census data from 2001 is largely the same as that from 2006 with the main 
difference being Mother’s level of education. In 2001, the variable was Mother’s years of 
education. In 2006, the variable is now Mother’s level of education. 


Once data is available, the following analysis will be done: 
e Jurisdiction by Mother’s level of education 

e Jurisdiction by Avg. family income/LICO 

e Jurisdiction by rent/own 


The SES variables will then be compared to GLA to see if any patterns emerge. The following 
tables will show these results: 

e GLA by Mother’s level of education 

e GLA by avg. family income/LICO 

e GLA by rent/own 
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